-
Notifications
You must be signed in to change notification settings - Fork 20
[FSSDK-12148] Event retry adjustment #398
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
base: master
Are you sure you want to change the base?
[FSSDK-12148] Event retry adjustment #398
Conversation
muzahidul-opti
left a comment
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Looks good, made a couple of comments
| /// Default constructor using the shared static HttpClient. | ||
| /// </summary> | ||
| static HttpClientEventDispatcher45() | ||
| public HttpClientEventDispatcher45() : this(null) |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Why null ? Relevalent comment could be made.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
This is constructor chaining in C#. this(null) will call the internal HttpeClientEventDispatcher45(HttpClient httpClient) which accepts httpClient as argument.
| if (shouldRetry && attemptNumber < maxAttempts - 1) | ||
| { | ||
| await Task.Delay(backoffMs).ConfigureAwait(false); | ||
| backoffMs = Math.Min(EventRetryConfig.MAX_BACKOFF_MS, |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Could you please cross check the backoff calculations?
backoffMs = min(maxBack, backOffMs * pow(BACKOFF_MULTIPLIER, attempNumber))
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Yeah. Both derives same result. 200, 400, 800 etc...
|
|
||
| if (shouldRetry && attemptNumber < maxAttempts - 1) | ||
| { | ||
| Thread.Sleep(backoffMs); |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Why don't we implement like the previous one.
await Task.Delay(backoffMs).ConfigureAwait(false);
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
So WebRequestEventDispatcher is for old platforms (.NET3.5, .NET4.0) and Task is not supported in 3.5 and for 4.0 no async/await support.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Uggg. We need to get <4.8 off our backs.
| if (shouldRetry && attemptNumber < maxAttempts - 1) | ||
| { | ||
| Thread.Sleep(backoffMs); | ||
| backoffMs = Math.Min(EventRetryConfig.MAX_BACKOFF_MS, |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
See the commnets☝🏻
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Yeah! Same as the formula.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Pull request overview
This pull request introduces a standardized retry and exponential backoff mechanism for event dispatching across the Optimizely C# SDK. The changes implement automatic retries (up to 3 total attempts) with exponential backoff (starting at 200ms, doubling to max 1000ms) for failed event dispatches on 5xx server errors and network failures.
Changes:
- Created
EventRetryConfigclass with centralized retry constants (max 2 retries, 200ms initial backoff, 1000ms max backoff, 2.0x multiplier) - Refactored
WebRequestEventDispatcher35andHttpClientEventDispatcher45to implement retry logic with exponential backoff - Updated
OdpEventManagerto use the newEventRetryConfigconstants and enhanced error logging - Removed
MAX_RETRIESconstant from ODPConstantsclass in favor of centralized configuration - Added comprehensive unit tests for
HttpClientEventDispatcher45covering success, retry, and backoff scenarios
Reviewed changes
Copilot reviewed 13 out of 13 changed files in this pull request and generated 6 comments.
Show a summary per file
| File | Description |
|---|---|
| OptimizelySDK/Event/EventRetryConfig.cs | New configuration class defining retry constants (max retries, backoff timing) |
| OptimizelySDK/Event/Dispatcher/HttpClientEventDispatcher45.cs | Refactored dispatcher with retry logic and exponential backoff for .NET 4.5+ |
| OptimizelySDK/Event/Dispatcher/WebRequestEventDispatcher35.cs | Refactored dispatcher with retry logic for .NET 3.5/4.0 |
| OptimizelySDK/Odp/OdpEventManager.cs | Updated to use EventRetryConfig constants and improved error logging |
| OptimizelySDK/Odp/Constants.cs | Removed MAX_RETRIES constant (moved to EventRetryConfig) |
| OptimizelySDK.Tests/EventTests/HttpClientEventDispatcher45Test.cs | New comprehensive test suite for retry behavior and backoff timing |
| OptimizelySDK.Tests/OdpTests/OdpEventManagerTests.cs | Updated test timeout to accommodate backoff delays |
| Various .csproj files | Added EventRetryConfig.cs to project configurations |
💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.
| var data = responseReader.ReadToEnd(); | ||
| // Success - exit the retry loop | ||
| return; | ||
| } |
Copilot
AI
Jan 20, 2026
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
If the response status code is not HttpStatusCode.OK (e.g., 201 Created, 204 No Content, or any 4xx error that doesn't throw an exception), the code falls through without returning or incrementing attemptNumber, creating an infinite loop. The code should either handle non-OK success codes (2xx), or increment attemptNumber and continue the loop, or return after logging.
| } | |
| } | |
| else | |
| { | |
| // Handle non-OK responses without exceptions using the same retry logic. | |
| var shouldRetry = ShouldRetry(response.StatusCode); | |
| if (shouldRetry && attemptNumber < maxAttempts - 1) | |
| { | |
| Thread.Sleep(backoffMs); | |
| backoffMs = Math.Min(EventRetryConfig.MAX_BACKOFF_MS, | |
| (int)(backoffMs * EventRetryConfig.BACKOFF_MULTIPLIER)); | |
| attemptNumber++; | |
| } | |
| else | |
| { | |
| LogMessage( | |
| LogLevel.ERROR, | |
| string.Format( | |
| "Error Dispatching Event after {0} attempt(s): received HTTP status code {1}", | |
| attemptNumber + 1, | |
| (int)response.StatusCode)); | |
| return; | |
| } | |
| } |
| var request = new HttpRequestMessage | ||
| { | ||
| RequestUri = new Uri(logEvent.Url), | ||
| Method = HttpMethod.Post, | ||
| // The Content-Type header applies to the Content, not the Request itself | ||
| Content = | ||
| new StringContent(json, Encoding.UTF8, "application/json"), | ||
| }; | ||
|
|
||
| foreach (var h in logEvent.Headers) | ||
| { | ||
| if (h.Key.ToLower() != "content-type") | ||
| foreach (var h in logEvent.Headers) | ||
| { | ||
| request.Content.Headers.Add(h.Key, h.Value); | ||
| if (h.Key.ToLower() != "content-type") | ||
| { | ||
| request.Content.Headers.Add(h.Key, h.Value); | ||
| } | ||
| } |
Copilot
AI
Jan 20, 2026
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
The HttpRequestMessage and its Content (StringContent) are not explicitly disposed. While HttpClient may handle the request disposal, the Content should be explicitly disposed to prevent resource leaks. Consider wrapping the request creation in a using statement or ensuring disposal in a finally block.
| // For non-HTTP exceptions, log and don't retry | ||
| Logger.Log(LogLevel.ERROR, $"Error Dispatching Event: {ex.GetAllMessages()}"); | ||
| return; | ||
| } |
Copilot
AI
Jan 20, 2026
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
The HttpResponseMessage is not disposed after use, which can lead to resource leaks. The response object should be disposed in a finally block or by using a using statement to ensure proper cleanup of HTTP connections.
| } | |
| } | |
| finally | |
| { | |
| response?.Dispose(); | |
| } |
| public class WebRequestClientEventDispatcher35 : IEventDispatcher | ||
| { | ||
| // TODO Catch and Log Errors | ||
| public Logger.ILogger Logger { get; set; } | ||
|
|
||
| private HttpWebRequest Request = null; | ||
| public ILogger Logger { get; set; } | ||
|
|
||
| /// <summary> | ||
| /// Dispatch the Event | ||
| /// The call will not wait for the result, it returns after sending (fire and forget) | ||
| /// But it does get called back asynchronously when the response comes and handles | ||
| /// Dispatch the Event with retry and exponential backoff. | ||
| /// The call will not wait for the result, it returns after sending (fire and forget) | ||
| /// But it does get called back asynchronously when the response comes and handles | ||
| /// </summary> | ||
| /// <param name="logEvent"></param> | ||
| public void DispatchEvent(LogEvent logEvent) | ||
| { | ||
| Request = (HttpWebRequest)WebRequest.Create(logEvent.Url); | ||
| ThreadPool.QueueUserWorkItem(_ => DispatchWithRetry(logEvent)); | ||
| } | ||
|
|
||
| Request.UserAgent = "Optimizely-csharp-SDKv01"; | ||
| Request.Method = logEvent.HttpVerb; | ||
| /// <summary> | ||
| /// Dispatch event with retry logic and exponential backoff. | ||
| /// </summary> | ||
| private void DispatchWithRetry(LogEvent logEvent) | ||
| { | ||
| var attemptNumber = 0; | ||
| var backoffMs = EventRetryConfig.INITIAL_BACKOFF_MS; | ||
| var maxAttempts = 1 + EventRetryConfig.MAX_RETRIES; | ||
|
|
||
| foreach (var h in logEvent.Headers) | ||
| while (attemptNumber < maxAttempts) | ||
| { | ||
| if (!WebHeaderCollection.IsRestricted(h.Key)) | ||
| HttpWebRequest request = null; | ||
| HttpWebResponse response = null; | ||
| try | ||
| { | ||
| Request.Headers[h.Key] = h.Value; | ||
| } | ||
| } | ||
| request = (HttpWebRequest)WebRequest.Create(logEvent.Url); | ||
| request.UserAgent = "Optimizely-csharp-SDKv01"; | ||
| request.Method = logEvent.HttpVerb; | ||
|
|
||
| Request.ContentType = "application/json"; | ||
| foreach (var h in logEvent.Headers) | ||
| { | ||
| if (!WebHeaderCollection.IsRestricted(h.Key)) | ||
| { | ||
| request.Headers[h.Key] = h.Value; | ||
| } | ||
| } | ||
|
|
||
| using (var streamWriter = new StreamWriter(Request.GetRequestStream())) | ||
| { | ||
| streamWriter.Write(logEvent.GetParamsAsJson()); | ||
| streamWriter.Flush(); | ||
| streamWriter.Close(); | ||
| } | ||
| request.ContentType = "application/json"; | ||
|
|
||
| var result = | ||
| Request.BeginGetResponse(new AsyncCallback(FinaliseHttpAsyncRequest), this); | ||
| } | ||
| using (var streamWriter = new StreamWriter(request.GetRequestStream())) | ||
| { | ||
| streamWriter.Write(logEvent.GetParamsAsJson()); | ||
| streamWriter.Flush(); | ||
| streamWriter.Close(); | ||
| } | ||
|
|
||
| private static void FinaliseHttpAsyncRequest(IAsyncResult result) | ||
| { | ||
| var _this = (WebRequestClientEventDispatcher35)result.AsyncState; | ||
| _this.FinalizeRequest(result); | ||
| } | ||
| response = (HttpWebResponse)request.GetResponse(); | ||
|
|
||
| private void FinalizeRequest(IAsyncResult result) | ||
| { | ||
| var response = (HttpWebResponse)Request.EndGetResponse(result); | ||
| if (response.StatusCode == HttpStatusCode.OK) | ||
| { | ||
| using (var responseStream = response.GetResponseStream()) | ||
| using (var responseReader = | ||
| new StreamReader(responseStream, Encoding.UTF8)) | ||
| { | ||
| responseReader.ReadToEnd(); | ||
| } | ||
|
|
||
| if (response.StatusCode == HttpStatusCode.OK) | ||
| { | ||
| // Read the results, even though we don't need it. | ||
| var responseStream = response.GetResponseStream(); | ||
| var streamEncoder = System.Text.Encoding.UTF8; | ||
| var responseReader = new StreamReader(responseStream, streamEncoder); | ||
| var data = responseReader.ReadToEnd(); | ||
| // Success - exit the retry loop | ||
| return; | ||
| } | ||
| } | ||
| catch (WebException ex) | ||
| { | ||
| var httpResponse = ex.Response as HttpWebResponse; | ||
| var shouldRetry = ShouldRetry(httpResponse?.StatusCode); | ||
|
|
||
| if (shouldRetry && attemptNumber < maxAttempts - 1) | ||
| { | ||
| Thread.Sleep(backoffMs); | ||
| backoffMs = Math.Min(EventRetryConfig.MAX_BACKOFF_MS, | ||
| (int)(backoffMs * EventRetryConfig.BACKOFF_MULTIPLIER)); | ||
| attemptNumber++; | ||
| } | ||
| else | ||
| { | ||
| LogMessage(LogLevel.ERROR, | ||
| string.Format("Error Dispatching Event after {0} attempt(s): {1}", | ||
| attemptNumber + 1, ex.GetAllMessages())); | ||
| return; | ||
| } | ||
| } | ||
| catch (Exception ex) | ||
| { | ||
| // For non-web exceptions, log and don't retry | ||
| LogMessage(LogLevel.ERROR, "Error Dispatching Event: " + ex.GetAllMessages()); | ||
| return; | ||
| } | ||
| finally | ||
| { | ||
| response?.Close(); | ||
| } | ||
| } | ||
| else | ||
| } | ||
|
|
||
| /// <summary> | ||
| /// Determines whether a request should be retried based on HTTP status code. | ||
| /// Retries on 5xx server errors and network failures (null status code). | ||
| /// </summary> | ||
| private static bool ShouldRetry(HttpStatusCode? statusCode) | ||
| { | ||
| // Retry on network failures (no response) | ||
| if (statusCode == null) | ||
| { | ||
| // TODO: Add Logger and capture exception | ||
| //throw new Exception(string.Format("Response Not Valid {0}", response.StatusCode)); | ||
| return true; | ||
| } | ||
|
|
||
| // Retry on 5xx server errors | ||
| var code = (int)statusCode.Value; | ||
| return code >= 500 && code < 600; | ||
| } | ||
|
|
||
| /// <summary> | ||
| /// Helper method to log messages safely when Logger might be null. | ||
| /// </summary> | ||
| private void LogMessage(LogLevel level, string message) | ||
| { | ||
| Logger?.Log(level, message); | ||
| } | ||
| } | ||
| } |
Copilot
AI
Jan 20, 2026
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
The WebRequestClientEventDispatcher35 class has been significantly refactored with new retry logic, but there are no tests covering this implementation. Since HttpClientEventDispatcher45 has comprehensive tests including retry scenarios, exponential backoff timing, and various error conditions, the WebRequestClientEventDispatcher35 should have equivalent test coverage to ensure the retry logic works correctly on .NET 3.5/4.0 platforms.
| foreach (var h in logEvent.Headers) | ||
| { | ||
| request.Content.Headers.Add(h.Key, h.Value); | ||
| if (h.Key.ToLower() != "content-type") | ||
| { | ||
| request.Content.Headers.Add(h.Key, h.Value); | ||
| } | ||
| } |
Copilot
AI
Jan 20, 2026
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
This foreach loop implicitly filters its target sequence - consider filtering the sequence explicitly using '.Where(...)'.
| var response = new HttpResponseMessage(statusCode) | ||
| { | ||
| Content = new StringContent("{}"), | ||
| }; |
Copilot
AI
Jan 20, 2026
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Disposable 'HttpResponseMessage' is created but not disposed.
mikechu-optimizely
left a comment
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Implementation looks good. Small suggestion below.
| Request.Headers[h.Key] = h.Value; | ||
| } | ||
| } | ||
| request = (HttpWebRequest)WebRequest.Create(logEvent.Url); |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Consider using as and checking for null over type coercion.
|
|
||
| if (shouldRetry && attemptNumber < maxAttempts - 1) | ||
| { | ||
| Thread.Sleep(backoffMs); |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Uggg. We need to get <4.8 off our backs.
| var _this = (WebRequestClientEventDispatcher35)result.AsyncState; | ||
| _this.FinalizeRequest(result); | ||
| } | ||
| response = (HttpWebResponse)request.GetResponse(); |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
as here too perhaps?
Summary
This pull request introduces a standardized retry and exponential backoff mechanism for event dispatching across the Optimizely SDKs. The changes ensure that failed event dispatches are retried up to three times with increasing delays, improving reliability in the face of transient network or server errors.
Test plan
Existing tests should pass
Issues